Rule Extraction, Fuzzy ARTMAP, and Medical Databases

نویسندگان

  • Gail A. Carpenter
  • Ah-Hwee Tan
چکیده

This paper shows how knowledge, in the form of fuzzy rules, can be derived from a. self-organizing supervised learning neural network called fuzzy ARTMAP. Rule extraction proceeds in two stages: pruning removes those recognition nodes whose confidence index falls below a selected threshold; and quantization of continuous learned weights allows the final system state to be translated into a usable set of rules. Simulations on a medical prediction problem, the Pima Indian Diabetes (PID) databa,se, illustrate the method. In the simulations, pruned networks about 1/3 the size of the original actually show improved performance. Quantization yields comprehensible rules with only slight degradation in test set prediction performance. Introduction: Fuzzy ARTMAP for Rule Extraction Fuzzy ARTMAP is a. neural network architecture that performs incrementa.! supervised learning of recognition categories ami multidimensional maps of both analog and binary patterns (Carpenter et a!., 1992). When performing classification tasks, fuzzy AHTMAP formulates recognition categories of input patterns, and associates each category with its respective prediction. For medical database analysis, a probabilistic predictive score is often more desirable than a yes-no answer. For such problems, mult,iple AHTMAPs can be trained on a single set of inputs, presented with di!Ierent orderings, and their combined voting score used to generate a probabilistic prediction. This approach has been applied successfully to a Denver VA Coronary Artery Bypass Grafting (CABG) data set (Goodman et a!., 1992). Rules can be derived from an ARTMAP network more readily than from a backpropagation network, in which the roles of hidden units are usually not explicit. In a. fuzzy ARTMAP network, each recognition node in theF-t field (Figure 1) roughly corresponds to a rule. Each node has an associated weight vector that can be directly translated into a verbal description of the corresponding rule. However, large databases typically cause ARTMAP to generate too many rules to be of practical use. The goal of the rule extraction task is thus to select a. small set of highly predictive recognition nodes and to describe them in comprehensible form. To evaluate a. recognition node, a confidence factor is computed that measures both usage and accuracy. Removal of low confidence recognition categories 1Support.ed in part by British Petroleum (llP-89-A-1204), DARPA (AFOSR-90-0083 and ONR-N0001492-J-4015), the National Science Foundation (NSF-IH.I-90-00530), and the Office of Naval Research (ONRN 00014-9 1-J -41 00). 2Supported by the Institute of Systems Science, National University of Singapore. Figure 1: Fuzzy AHTMAP architecture (Carpenter et a!., 1992). created by ad hoc examples produces smaller networks. In fact, this network pruning procedure can even improve test set performance by removing misleading special cases. In order to describe the knowledge in simplifted rule form, real-valued weights are quantized into a small set of values. Various rule extraction methods were evaluated using a Pima Indians Diabetes (PID) data set in which the predicting index is whether or not a patient shows signs of diabetes. Pruning produced rule sets that were consistenLly 1/3 the size of the original networks, and also produced superior test set performance. Quantization produced more comprehensible rules at only a slight cost in terms of performance. Rule Extraction Procedure Fuzzy An:rMAP consists of two fuzzy ART modules (Carpenter, Grossberg, and Rosen, 1991) connected by a map field (Figure 1). The map field forms predictive associations between AHTa and AHTb categories. Internal control rnechanisms realize the match tracking rule whereby the vigilance parameter of AR'I'a increases in response to a predictive mismatch at AR:l'b. This on-line error-correction procedure allows the system to function in a fast-learning mode. In classification tasks, each node in the ART a field P~ codes a. recognition category of AH'L input patterns. During training, each such node learns to predict an ART6 category. Learned weight vectors, one for each P~ node, thereby constitute a set of rules that link antecedents to con~equences (Figure 2). Pruning recognition categories: To reduce the complexity of fuzzy AllTMAP, a pruning procedure aims to select a small set of good rules from a. trained network. The algorithm evaluates each P~ recognition node in terms of its coding statistic in the training Prediction (Consequence) _______________ j Figure 2: Schematic diagram of a rule in fuzzy AHTMAP. Each J~ node maps a prototype feature vector to a prediction. v1 v2 v3 v4 vQ ~~~~---~~--=:1......-z~ =t..~--=:~~-.:::_~:-~l~=-=---:=J 0 1 Weight Value Figure 3: Quantization algorithms: Truncation (left) and round-off (right). Arrows indicate the direction of quantization. set and its predictive performance on a test set, resulting in a confidence factor, described below. Nodes with low confidence are pruned. Performance of the pruned system is then evaluated on a different test set. 'l'he pruning algorithm evaluates a.n Fi' recognition category .i in terms of a confidence factor C Fj:

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Rule Extraction: From Neural Representation Architecture to Symbolic

This paper shows how knowledge, in the form of fuzzy rules, can be derizted from a superuised learning neural network called fuzzy ARTMAP. Rule extraction proceeds in two stages: pruning, which simplifies the network structure by remooing excessiae recognition categories and weights; and quantization of continuous learned weights, which allows the final system state to be translated into a usab...

متن کامل

Cross-validation in Fuzzy ARTMAP for large databases

In this paper we are examining the issue of overtraining in Fuzzy ARTMAP. Over-training in Fuzzy ARTMAP manifests itself in two different ways: (a) it degrades the generalization performance of Fuzzy ARTMAP as training progresses; and (b) it creates unnecessarily large Fuzzy ARTMAP neural network architectures. In this work, we are demonstrating that overtraining happens in Fuzzy ARTMAP and we ...

متن کامل

Application of the fuzzy ARTMAP neural network model to medical pattern classification tasks

This paper presents research into the application of the fuzzy ARTMAP neural network model to medical pattern classification tasks. A number of domains, both diagnostic and prognostic, are considered. Each such domain highlights a particularly useful aspect of the model. The first coronary care patient prognosis, demonstrates the ARTMAP voting strategy involving 'pooled' decision-making using a...

متن کامل

Evaluation of Fuzzy ARTMAP with DBSCAN in VLSI Application

The various applications of VLSI circuits in highperformance computing, telecommunications, and consumer electronics has been expanding progressively, and at a very hasty pace. This paper describes a new model for partitioning a circuit using DBSCAN and fuzzy ARTMAP neural network. The first step is concerned with feature extraction, where we had make use DBSCAN algorithm. The second step is th...

متن کامل

Pipelining of Fuzzy ARTMAP without matchtracking: Correctness, performance bound, and Beowulf evaluation

Fuzzy ARTMAP neural networks have been proven to be good classifiers on a variety of classification problems. However, the time that Fuzzy ARTMAP takes to converge to a solution increases rapidly as the number of patterns used for training is increased. In this paper we examine the time Fuzzy ARTMAP takes to converge to a solution and we propose a coarse grain parallelization technique, based o...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 1992